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ABSTRACT 

This  paper  discusses  sense  making,  and  its  role  in  how  intelligence  analysts  understand  and 
interpret  events.  The  inherent  limitations  in  the  way  an  analyst,  based  on  organisational  and 
personal  perspectives,  understands  the  world  is  described  and  points  toward  the  need  for 
meta-sense  making  techniques.  Insights  from  complex  systems  theory  and  knowledge 
management  can  be  used  to  understand  how  techniques  from  these  areas  can  be  applied  in 
practice,  and  assist  in  mitigating  some  of  the  risks  due  to  the  cognitive  limitations  inherent  in 
intelligence  processes.  The  relation  of  an  augmented  capability  for  sensing  and  understanding 
to  emerging  whole-of-enterprise  concepts  such  as  Sense  and  Respond  and  UC2  is  also 
sketched. 
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Sense  Making  in  the  Australian  Defence 
Organisation  (ADO)  Intelligence  Community 

Executive  Summary 

Since  the  end  of  the  Cold  War  estimative  intelligence,  in  particular,  has  shifted  from 
dealing  with  a  knowable  and  reasonably  well-defined  objective  reality  to  one  in  which 
reality  is  fundamentally  complex,  uncertain,  and  in  some  cases  unknowable.  Gaining 
the  edge  over  an  adversary  now  relies  more  on  the  analytical,  predictive  and  cognitive 
abilities  that  can  be  brought  to  bear  on  the  mass  of  information  than  on  the  collection  of 
the  information  per  se. 

The  organisations  within  the  Australian  Intelligence  Community  (AIC)  are  viewed  as 
contexts  for  sense  making  of  complex  and  uncertain  environments.  Sense  making  occurs 
in  novel  situations  where  there  is  a  divergence  between  what  is  expected  and  what  is 
observed.  When  dealing  with  these  environments,  the  cognitive  challenges  facing 
analysts  highlight  the  need  for  a  good  understanding  within  the  AIC  of  the  underlying 
mechanisms  that  allow  for  growth  and  contestability  of  knowledge  in  complex  and 
uncertain  areas. 

It  is  argued  that  the  way  analysts  understand  the  world  is  heavily  influenced  by  their 
own  and  organisational  perspectives.  Complex  systems  theory  can  be  used  to  gain  new 
insights  of  real  world  events  and  our  ability  to  understand  them. 

Core  to  improving  the  cognitive  activity  of  intelligence  analysis  are  tools  for  hypothesis 
testing  and  for  "what-if"  scenario  testing,  as  is  a  better  understanding  of  semantic 
computing  and  the  use  of  service  oriented  architectures  to  support  whole-of -enterprise 
information  management. 

Collaborative  structures  that  relate  to  the  formation  of  new  patterns  of  perceptions, 
new  ways  of  understanding  the  world,  and  the  disruption  of  existing  mind-sets  are 
needed.  Such  structures  seek  to  tap  into  breadth  of  knowledge  in  and  beyond  the  ADO 
intelligence  community  to  create  an  environment  encouraging  counter-intuitive 
insights. 
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1.  Introduction 


Despite  continuous  improvements  in  the  technology  used  to  collect  and  process  raw 
intelligence  data,  intelligence  is  still  largely  a  cognitive  activity  -  an  activity  rich  in 
decision  making,  interpretation  and  perceptions.  And  while  there  have  been 
considerable  improvements  in  addressing  the  technological  aspects  of  intelligence 
processing  and  gathering,  there  haven't  been  comparable  improvements  in  addressing 
the  fundamental  cognitive  challenges  intelligence  analysts  face  (Heuer  1999).  Included 
among  these  cognitive  issues  are  the  following: 

•  The  mind  is  poorly  "wired"  to  deal  effectively  with  many  forms  of  uncertainty 
that  surround  complex,  indeterminate  intelligence  issues  and  the  "fog" 
associated  with  denial  and  deception  operations; 

•  Increased  awareness  of  cognitive  and  other  "unmotivated"  biases,  such  as  the 
tendency  to  see  information  confirming  an  already-held  judgment  more  vividly 
than  one  sees  "  discontinuing"  information,  does  little  by  itself  to  help  analysts 
deal  effectively  with  uncertainty1; 

•  Tools  and  techniques  that  gear  the  analyst's  mind  to  apply  higher  levels  of 
critical  thinking  can  substantially  improve  analysis  on  complex  issues  for  which 
information  is  incomplete,  ambiguous,  contradictory,  and  often  deliberately 
distorted.  Key  examples  of  such  intellectual  devices  include  techniques  for 
structuring  information,  challenging  assumptions,  and  exploring  alternative 
interpretations. 

The  goal  of  this  paper  is  to  explore  the  cognitive  dimension  of  intelligence  analysis,  and 
in  particular  to  provide  a  conceptual  basis  for  understanding  the  cognitive  dimension 
of  intelligence  processing  and  how  these  insights  can  be  exploited  for  better 
intelligence  assessments. 

This  paper  discusses  sense  making,  and  its  role  in  how  intelligence  analysts 
understand  and  interpret  events.  The  inherent  limitations  in  the  way  an  analyst 
understands  based  on  organisational  and  personal  perspectives  is  described,  pointing 
towards  the  need  for  meta-perception  or  meta-sense  making  techniques.  Insights  from 
complex  systems  theory  and  knowledge  management  can  be  used  to  understand  how 
some  of  these  techniques  can  be  applied  in  practice,  and  assist  in  mitigating  some  of 
the  risks  due  to  the  cognitive  limitations  inherent  in  the  intelligence  processes. 

How  these  ideas  fit  into  the  broader  military  enterprise  are  also  sketched,  in  particular 
the  relation  to  adaptive  operational  planning  and  Command  and  Control. 


1  Paul  Dibb  [The  Australian  Newspaper,  24  Feb  04],  when  talking  about  the  Australian  intelligence 
relating  to  the  search  for  WMD  in  Iraq  before  the  2003  invasion,  makes  the  point  that  when  judging  hard 
issues  intelligence  agencies  are  prone  to  look  for  confirming  rather  than  disconfirming  evidence. 
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2.  Sense  Making  in  a  Complex  World 

2.1  Sense  Making 

Sense  making  is  a  process  that  intelligence  analysts  and  decision  makers  go  through 
when  dealing  with  complex  problems  involving  ambiguous  or  poor  information, 
changing  circumstances  and  multiple  players.  Sense  making  is  the  process  of  finding 
the  'best'  set  of  perceptions  that  helps  to  understand  the  problem/  situation. 

Much  has  been  written  about  sense  making  recently2.  For  example  Weick  (1995)  talks 
about  a  sense  making  mechanism  with  three  elements  -  events  or  objects,  a  framework 
and  a  relation  between  them.  In  addition  to  placing  items  into  frameworks,  sense 
making  is  described  in  terms  of  comprehension  and  construction  of  meanings,  and  is 
contrasted  by  Weick  with  interpretation.  From  this  viewpoint  interpretation,  loosely 
defined  as  an  acceptable  and  approximate  translation  (usually  of  some  sort  of  text),  is  a 
component  of  sense  making.  But  sense  making  addresses  how  the  text  is  constructed 
as  well  as  how  it  is  read.  It  often  involves  the  collective  application  of  intuition  and 
largely  "tacit"  knowledge  based  on  experience  and  expertise  in  a  demanding,  difficult 
or  complex  task.  For  instance  Tovstiga  et  al.  (2004)  explore  sense  making  and  learning 
in  a  string  quartet,  and  draw  strong  parallels  between  the  quartet  and  complex 
organisations. 

Within  complex  environments,  we  view  sense  making  as  a  precursor  to  situation 
awareness  (SA),  and  in  many  ways,  as  a  driver  of  the  process  of  SA.  Endsley's 
definition  of  SA  (Endsley  1995)  -  "Situation  awareness  is  the  perception  of  the  elements 
in  the  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their 
meaning,  and  the  projection  of  their  status  in  the  near  future"  -  introduces  and  relates 
the  three  elements  of  SA:  perception,  comprehension  and  projection.  A  sense  making 
framework  can  impact  each  of  these  elements  -  it  can  tell  you  what  to  look  for  as  a 
precursor  to  perception,  it  impacts  how  perceived  events  or  objects  appear  to  relate  to 
each  other  (known  as  situation  assessment),  and  how  these  in  turn  project  onto  their 
potential  or  future  effects  in  this  or  another  arena  (known  as  impact  assessment). 

Each  of  these  SA  elements  affects  the  other  in  non-linear  ways,  and  emphasises  the 
need  for  a  sense  making  framework,  particularly  when  the  stakes  are  high.  An  example 
is  the  events  described  in  the  film  Fog  Of  War  where  Defense  Secretary  McNamara 
recounts  an  incident  in  1964  when  two  US  destroyers  (the  Maddox  and  the  Turner  Joy) 
appeared  to  have  been  attacked  by  torpedoes.  A  few  days  before  this  the  Maddox,  on 
an  intelligence  gathering  operation  in  the  Gulf,  had  been  attacked  by  P-4  torpedo  boats 
in  a  battle  lasting  22  minutes,  and  this  latest  episode  was  assumed  to  be  an 
intensification  of  hostilities  to  test  the  US's  resolve  to  escalate  the  conflict  to  new  levels. 


2  For  instance  a  Command  and  Control  Research  Program  Sensemaking  symposium  in  2001  (Leedom 
200 1 )  identified  a  greater  need  for  understanding  the  foundations  of  sense  making,  particularly  in  light  of 
new  operating  concepts  such  as  EBO  and  NCW. 
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Figure  1  shows  a  simplified  Endsley-type  view  of  the  incident  put  together  by  the 
authors. 


perception  of 
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Figure  1:  An  Endsley  (situation  awareness)  representation  of  the  Gulf  of  Tonkin  incident,  1964 

At  the  object  assessment  level  there  was  some  debate  about  whether  the  radar  signals 
actually  indicated  attacking  torpedo  boats,  but  this  was  resolved  in  the  affirmative, 
partly  one  suspects  because  it  fitted  with  the  previous  incident.  At  the  comprehension 
and  projection  levels  there  was  agreement  between  the  Pentagon  and  President 
Johnson  that  the  US  must  respond  quickly  to  avoid  losing  face.  The  end  result  was  that 
the  US  Congress  quickly  passed  the  Gulf  of  Tonkin  resolution  that  paved  the  way  for  a 
massive  escalation  in  the  war  against  Vietnam.  It  emerged  that  there  were  no  torpedo 
boat  attacks,  but  the  mindset  of  the  US  military  and  policy  makers  did  not  allow  for  a 
critical  examination  of  the  initial  evidence,  and  there  was  a  lack  of  diversity  in  the 
military  sense  making  apparatus  to  allow  better  assessments  to  be  made. 

Another,  structurally  different,  example  is  in  the  Cuban  Missile  Crisis  when  the  Soviet 
ship  Grozny  crossed  the  blockade  imposed  by  the  US  Navy  around  Cuba  (see  Figure 
2).  Here  the  perception  was  spot-on  -  the  ship  had  indeed  crossed  the  blockade  -  but 
the  comprehension  and  projection  of  this  incident  was  very  different  for  the  Pentagon 
in  the  form  of  Admiral  Anderson  and  the  policy  maker  McNamara.  Though  each 
behaved  rationally  and  professionally  they  had  different  sense  making  frameworks 
with  which  to  process  the  underlying  event  information,  and  came  up  with  different 
understandings  of  the  action  to  take. 
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Figure  2:  An  Endsley  (situation  awareness)  representation  of  a  blockade  incident  in  the  Cuban 

Missile  Crisis,  1962 

The  inclusion  of  acts  -  such  as  "Attack  the  Grozny"  -  in  these  diagrams  is,  strictly 
speaking,  outside  of  Endsley's  cognitive  model  of  situation  awareness.  It  could  also  be 
argued  that,  from  the  perspective  of  the  Commander  of  one  of  the  ships  in  the  Gulf  of 
Tonkin,  the  "tracks"  on  the  radar  screen  are  at  the  perception  (object)  level,  leading 
through  to  an  impact  assessment  that  the  ship's  anti-patrol  boat  defences  need  to  be 
heightened.  This  indicates  that  SA  is  highly  subject  dependent,  and  the  figures 
represent  a  conflation  of  the  SA  of  a  number  of  key  players  in  the  incidents. 

It  is  also  possible  to  represent3  these  incidents  using  the  other  main  abstractions  of 
military  decision-making,  namely  Boyd's  OODA  loop  and  Klein's  RPD  (Recognition- 
Primed  Decision)  model  (Klein  1993),  which  include  decision  and  (for  OODA)  action 
phases.  In  complex  situations  Conklin  (2004)  believes  that  the  cognitive  processes  of 
sensing,  understanding,  deciding  and  acting  (or  their  equivalents  in  the  major  models) 
are  not  performed  linearly.  To  use  a  mechanistic  analogy,  there  are  feed  forward  and 
feedback  loops,  and  processes  going  on  in  parallel  and  apparently  out  of  order  that  are 
not  represented  in  these  simple  models.  Lambert  and  Scholz  (2005)  discuss  in  detail 
how  the  ADF  JMAP  (Joint  Military  Appreciation  Process),  which  is  based  on  OODA, 
departs  in  practice  from  the  linear  model. 


3  And  it  is  possible  to  inter-relate  all  three  models  -  see  Rousseau  and  Breton  2005. 
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2.2  The  Intelligence  Conundrum 

The  point  to  be  made  here  is  that  whatever  model  is  assumed,  for  complex  or  uncertain 
environments  sense  making  is  an  implicit  and  organic  part  of  the  process  that  ascribes 
meaning  to  perceived  sensations.  This  paper  attempts  to  open  up  some  of  the 
mechanisms  for  sense  making  of  complex  issues  at  an  enterprise  level  for  the 
intelligence  community. 

In  dealing  with  multi-faceted  and  complex  issues  analysts  have  always  been 
challenged  in  providing  sound  and  timely  advice.  Estimative  intelligence,  which  is 
concerned  with  what  might  be  or  what  might  happen,  has  increased  in  difficulty  and 
changed  in  nature  since  the  end  of  the  Cold  War  where  intelligence  knew  the  problem 
and  could  envisage  a  reality  it  was  seeking  to  comprehend.  Joseph  Nye's  phrase 
"Mysteries  Not  Puzzles"  (Nye  1994)  reflects  the  mind  shift  from  a  knowable  objective 
reality  to  one  in  which  reality  is  fundamentally  complex,  uncertain,  and  in  some  cases 
unknowable4. 

The  Australian  intelligence  enterprise  is  no  different.  The  growth  in  the  multiplicity, 
diversity  and  type  of  security  threats  facing  Australia  point  toward  the  need  for  greater 
adaptability  and  innovation  from  the  intelligence  enterprise  in  meeting  these  threats. 
And,  as  a  consequence  of  the  information  technology  revolution  over  the  last  20  years, 
the  breadth  and  depth  of  data  to  be  considered  has  expanded  enormously,  and  will 
continue  to  expand.  Access  to  technology  such  as  high-speed  communication  links, 
encryption  and  powerful  computers  is  becoming  ubiquitous  and  having  these  no 
longer  provides  a  clear  operational  edge  for  the  community.  The  depth  of 
knowledge/expertise  required  to  analyse,  anticipate  and  understand  complex,  multi¬ 
variate  problems  has  also  grown  considerably.  The  edge  over  an  adversary  is  now 
more  in  the  analytical,  predictive  and  cognitive  abilities  that  can  be  brought  to  bear  on  the 
mass  of  information  than  on  the  collection  of  the  information  per  se.  This  has  important 
capability  implications  for  developing  a  balance  between  sensing  (capacity  to  observe) 
and  sense  making  (capacity  to  orientate)  in  the  context  of  the  ADO's  migration  towards 
network-enabled  operations. 

A  key  to  sense  making  is  attempting  to  appreciate  the  understanding  that  humans 
bring  to  the  process,  and  the  way  in  which  that  understanding  is  used,  shared,  tested 
and  evolved  during  the  process.  How  sense  making  occurs,  and  how  understanding  is 
used,  is  strongly  dependent  on  how  we  think  and  how  we  represent  the  world.  This  is 


4  There  is  a  critical  distinction  between  what  may  be  called  conventional  threats  and  emerging  threats. 

[ref:  Robert  Steele,  On  Intelligence  -  Spies  and  Secrecy  in  an  Open  World,  OSS,  2001,  quoting  from  Gen. 
Alfred  Gray,  Commandant  of  the  Marine  Corps.,  Global  Intelligence  Challenges  in  the  1990’s,  American 
Intelligence  Journal  1989-1990]  Conventional  threats  are  associated  with  static  orders  of  battle,  linear 
development  and  deployment  capabilities,  and  well-understood  rules  of  engagement  and  doctrine.  The 
emerging  threat,  in  contrast,  is  non-governmental,  non-conventional,  dynamic  or  random,  and  non-linear 
with  no  constraints  or  predictable  doctrine. 
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the  territory  of  philosophy,  and  constitutes  a  framework  or  methodology  in  which 
knowledge  can  be  used. 

2.3  Knowledge  and  Understanding 

A  dictionary  definition  of  understanding  is  "the  faculty  of  thinking,  reasoning,  and 
acquiring  and  applying  knowledge".  It  often  relates  to  having  cognisance  of  a 
comprehensive,  interrelated  system  that  can  be  used  for  prediction  of  the  future  states 
of  a  system,  explanation  as  to  why  the  system  behaves  in  the  way  it  does,  and  how 
different  knowledge  relating  to  it  fits  together.  However,  an  expert  juggler  who  can 
adapt  to  different  objects  to  juggle  or  different  partners  to  juggle  with  might  be  said  to 
understand  juggling,  though  the  knowledge  on  which  this  is  based  is  largely  tacit  and 
based  much  more  on  experience  than  any  model,  system  or  theory.  They  might,  for 
instance,  find  it  difficult  to  explain  how  to  juggle.  This  leads  to  a  somewhat  broader 
view  of  understanding  as  the  ability  to  successfully  apply  and  acquire  knowledge  for 
new  contexts. 

How  knowledge  is  defined  determines  how  it  is  managed,  so  it  is  important  to  have  a 
clear  definition.  And  the  terms  knowledge  and  information  are  so  broad  that  they 
become  meaningless  unless  given  clear  definition.  For  the  purposes  of  exploring  the 
role  that  knowledge  plays  in  sense  making  we  adopt  the  following  definition: 

"Knowledge  is  a  fluid  mix  of  framed  experience,  values,  contextual 
information,  and  expert  insight  that  provides  a  framework  for  evaluating  and 
incorporating  new  experiences  and  information.  It  originates  and  is  applied  in 
the  minds  of  knowers.  In  organizations,  it  often  becomes  embedded  not  only  in 
documents  or  repositories  but  also  in  organizational  routines,  processes, 
practices,  and  norms"  (Davenport  and  Prusak  1997). 

This  definition  ties  knowledge  closely  to  cognition  and  argues  that  knowledge  is  a 
dynamic,  conceptual  and  largely  human  process  of  organising,  re-organising  and 
making  sense  of  information,  experiences,  and  events  (Nitecki  1985).  In  this  view, 
knowledge  is  not  a  "thing"  or  a  system,  but  an  ephemeral,  active  cognitive  process  of 
relating  (Stacey  1992)  the  external  world  and  external  events  to  what  is  already  known. 

Some  knowledge  can  be  made  explicit,  and  codified  as  information.  A  key 
characteristic  of  this  type  of  knowledge  is  that  it  can  be  disembodied  from  its  knower 
or  originator.  Since  this  type  of  knowledge  can  be  encoded  as  information  and  can  be 
disembodied  from  its  creator,  it  can  be  shared  as  words,  pictures,  e-mails,  web  pages, 
text  guides,  instruction  manuals,  and  so  on,  and  can  be  held  in  forms  that  can  be 
duplicated,  shared,  or  stored  in  a  computer  system.  Managing  this  kind  of  explicit 
knowledge  is  the  realm  of  information  management  methods  and  information 
management  technology. 
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In  contrast  to  explicit  knowledge,  tacit  knowledge  is  the  intangible,  internal,  and 
intuitive  knowledge  in  the  human  mind.  By  its  very  nature,  it  is  very  difficult  to  make 
tacit  knowledge  explicit  (Polanyi  1966).  Tacit  knowledge  has  to  do  with  insight, 
understanding,  experience,  capability,  skill  and  expertise.  This  kind  of  knowledge  can 
be  shared  through  an  ongoing  interaction  between  the  holder  (the  knower)  and  the 
receiver  of  the  knowledge. 

This  distinction  between  tacit  and  explicit  knowledge  is  important  for  the  intelligence 
community  because  the  types  of  knowledge  important  for  sense  making  -  expertise, 
insight,  experiences,  highly  refined  analytical  skill  -  are  likely  to  be  tacit  and  hence 
impossible  to  disembody  from  the  individual  members  of  the  intelligence  community 
who  hold  them.  This  has  significant  implications  for  developing  a  knowledge-centric 
capability  within  the  intelligence  enterprise;  this  re-focusing  has  considerable 
consequences  for  organisation  and  culture  beyond  the  present  emphasis  on 
information  technology. 

One  of  the  great  myths  of  knowledge  management  is  that  knowledge  of  this  type  can 
be  readily  converted  into  information,  and  is  thus  open  to  exploitation  by  information 
management  techniques.  By  definition,  however,  tacit  knowledge  cannot  be  converted 
directly  to  explicit.  And  in  the  intelligence  domain  in  particular,  knowledge  in  support 
of  decision-superiority  is  very  context-specific.  This  makes  knowledge  difficult  if  not 
impossible  to  articulate  without  being  prompted  by  a  specific  need  (McDermott  1999). 

Figure  3  shows  our  conceptualisation  of  the  relation  between  knowledge,  context, 
information  and  data  to  sense  making.  Note  that  this  differs  from  the  standard  data- 
information-knowledge-wisdom  pyramid  often  cited  in  the  knowledge  management 
literature. 


Data  - 

Information  — 

(Messages  with  potential 
to  inform) 


Context 

(Constraints,  intent) 


Knowledge 

(Understanding  based  on 
expertise,  skill  and  experience) 


Insight 
Meanin 
Judgement 


Figure  3:  A  schematic  relating  knowledge,  context,  information  and  data  to  sense  making 
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The  first  part  of  Davenport  and  Prusak's  (1997)  definition  focuses  on  knowledge  as 
something  that  resides  within  the  mind  of  the  knower  and  as  a  framework  for 
comprehension.  When  we  talk  about  knowledge  in  this  paper  this  is  what  we  are 
referring  to.  Since  knowledge  (by  this  definition)  is  in  the  mind  of  the  knower, 
knowledge  within  the  intelligence  community  exists  within  the  individual  members  of 
the  intelligence  community.  It  is  their  mix  of  experiences,  values,  analytical  skills,  and 
expert  insights  that  provide  the  intelligence  community  the  means  of  understanding, 
interpreting,  and  exploiting  the  information  it  collects  to  produce  intelligence  products. 

As  well  as  being  what  we  know,  knowledge  also  acts  as  a  framework  for  evaluating 
and  incorporating  new  experiences  and  information  (Davenport  and  Prusak  1997).  In 
essence,  we  use  our  existing  knowledge  to  create  our  new  knowledge.  New  events, 
experiences,  and  information  are  filtered  through  the  results  of  past  thinking,  patterns 
of  interpretation,  implicit  assumptions,  and  beliefs  that  have  been  learnt  and  built  from 
experience,  past  thinking  and  reflection  (Snowden  2002). 

For  the  intelligence  community,  filtering  new  experiences  and  information  through 
existing  patterns  of  interpretation,  implicit  assumptions,  and  beliefs  has  two  very 
significant  effects.  First,  it  affects  how  the  intelligence  community  interprets  the 
information  it  collects;  it  underpins  an  analyst's  interpretation  of  current  events, 
predictions  and  assessments  of  future  events.  Second,  it  influences  what  is  considered 
important,  relevant  and  worthy  of  consideration  within  the  intelligence  domain.  In 
essence,  it  drives  where  the  intelligence  community  looks  for  potential  problems  or  events  of 
interest,  and  how  it  comprehends  them  in  context. 

As  a  result,  it  is  critical  to  recognise  that  the  knowledge  important  to  the  intelligence 
community  -  the  expertise,  insight,  experiences,  highly  refined  analytical  skill  -  cannot 
be  easily  codified,  and  cannot  be  shared  as  information.  As  a  consequence,  the  key  to 
making  the  knowledge  resident  in  the  intelligence  community  more  productive  is  to 
provide  a  sound  methodology  for  thinking  and  to  place  greater  emphasis  on  the 
relationships  and  networks  between  staff  to  enable  knowledge  to  grow,  be  tested  and 
used  most  effectively. 

2.4  Complexity 

In  recent  years  the  cross-disciplinary  subject  of  Complexity  Science  has  opened  up  new 
understanding  in  many  fields  such  as  economics,  biology,  physics  and  computer 
science.  A  Complex  System  is  any  system  that  involves  a  large  number  of  dynamically 
interacting  elements5.  These  go  through  processes  of  change  that  are  not  describable  by 
a  single  rule  nor  are  reducible  to  only  one  level  of  explanation;  these  levels  often 
include  features  whose  emergence  cannot  be  predicted  from  their  current 


5  A  complex  system  is  differentiated  from  a  complicated  system  such  as  an  aircraft  that  can  be  de¬ 
composed  into  a  number  of  sub-systems  with  known  components,  relationships  and  function. 
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specifications.  The  field  of  Complexity  Theory  attempts  to  apply  scientific  methods  to 
these  complex  systems,  concentrating  more  on  the  interactions  and  dynamics  of  the 
system  than  the  elements  themselves. 

The  science  of  complexity  takes  two  phenomena  as  its  foundation: 

•  There  is  non-proportionality  in  complex  systems  because  small  changes  can 
have  large  consequences. 

•  There  is  non-additivity  because  the  whole  is  more  than  the  sum  of  its  parts. 

The  dominant  paradigm  of  complexity  science  is  one  of  non-linearity.  In  contrast  much 
of  traditional  science  deals  with  linear  systems  where  changes  in  the  system  input 
result  in  proportional  changes  in  the  output  (linearity),  the  whole  is  the  sum  of  its  parts 
(proportionality),  the  system  may  be  understood  by  observing  the  behaviour  of  its 
individual  parts  (reductionism),  and  the  system  has  processes  that  flow  along  orderly 
and  predictable  paths  with  clear  beginnings  and  rational  ends  (determinism). 

For  the  intelligence  community  sense  making  increasingly  involves  dealing  with,  and 
gaining  some  workable  understanding  of,  inherently  complex,  adaptive  and  interlinked 
systems,  such  as  social  and  military  organisations,  economies  and  public  opinion. 
Cause  and  effect  linkages  are  not  inherently  knowable  in  such  systems,  and  order  tends 
to  be  an  emergent  property  of  the  system  rather  than  a  fundamental  one.  Sense  making 
is  the  process  of  choosing  the  right  set  of  perceptions  and  mental  models  to  be  able  to  understand 
and  act  successfully  in  this  type  of  environment. 

2.5  Social  Complexity 

Snowden  (2005)  contrasts  the  complexity  of  the  systems  modelled  mathematically  by 
Complexity  Science  pioneers  such  as  Axelrod,  Kauffman  et  al.  (see  for  example 
Kauffman  1996)  with  the  complexity  of  human  systems.  Snowden  lists  the  unique 
aspects  of  human  systems  as  follows: 

•  Humans  make  decisions  based  on  patterns 

•  Humans  create  and  maintain  multiple  identities 

•  Humans  ascribe  intentionality  and  cause  where  none  necessarily  exists 

•  Humans  have  learnt  how  to  structure  their  social  interactions  to  create  order 

The  Cynefin  framework  (Kurtz  and  Snowden  2003)  takes  a  social  complexity-based 
approach  to  knowledge  management  and  sense  making.  It  maps  out  four  domains  of 
organisational  knowledge  (as  shown  in  Figure  4)  over  two  broad  areas  -  order  (RHS  of 
diagram)  and  un-order  (LHS). 

The  ordered  domain  is  characterised  by  complicated  (but  not  complex)  systems  with  a 
definite  known  or  knowable  order,  and  cause  and  effect  relations  that  can  be  studied 
and  discovered.  It  is  further  sub-divided  into  empirically  knowable  (the  realm  of 
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science,  the  expert  and  good  practice)  and  known  (the  realm  of  bureaucracy,  rules, 
procedures  and  best  practice). 


Complex  \ 

Pattern  management  1 

Matriarchal/Patriarchal  1 

leadership  1 

Probe,  sense,  respond  1 

Knowable 

Analytical/Reductionist 
Oligarchic  leadership 

Sense  and  respond 

V  • 

Chaos 

Known 

Turbulent  and  unconnected  J 

Legitimate  best  practice 

Charismatic  or  tyrannical  j 

Feudal  leadership 

leadership  I 

Categorise  and  respond 

Act,  sense,  respond  / 

Figure  4:  Cynefin  Framework 


In  contrast  sense  making  in  the  un-ordered  domain  recognises  the  inherent  complexity 
of  most  things  and  that  cause  and  effect  cannot  be  discovered  because  they  are  so 
closely  intertwined.  The  word  un-order  does  not  mean  a  lack  of  order,  it  means  a 
different  kind  of  order,  one  that  is  in  contrast  to  ordered-systems  thinking.  Ordered- 
systems  thinking  assumes  that  through  study  and  analysis  we  can  discover  empirically 
verifiable  general  rules  or  hypotheses  that  create  a  body  of  reliable  and  enduring 
knowledge.  The  domain  of  un-order  has  the  characteristics  of  complex,  adaptive 
systems;  here  the  whole  is  never  the  sum  of  the  parts,  end  points  are  highly  path 
dependent,  and  cause  and  effect  can  be  made  out  often  only  in  retrospect. 


In  the  ordered  domain  the  conditions  for  the  emergence  of  meaning  are  rationalist  and 
reductive;  the  processes  involve  sense  and  respond  for  the  knowable  and  categorise  and 
respond  for  the  known.  In  the  un-ordered  domain  new  science  and  new  approaches  are 
required  to  facilitate  the  use  of  knowledge  and  the  emergence  of  meaning.  Stacey 
(1992)  develops  this  perspective  by  drawing  on  complexity  sciences  as  the  field  for 
analogies  with  human  action.  This  alternative  perspective  places  self-organising 
interaction  at  the  centre  of  the  knowledge  creating  process  in  organisations. 


It  can  be  argued  that  the  intelligence  community  is  a  product  of  an  ordered-domain  view 
of  the  world  that  is  suited  to  the  types  of  threats  and  the  intelligence  environment 
prevalent  in  the  latter  half  of  the  last  century.  With  well-defined  threats  and  a  focus  on 
collection  and  technical  expertise,  the  community  naturally  aligned  itself  with  the 
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ordered  model  that  permits  exploitation  of  resources  within  the  community  for 
decision-makers  faced  with  well-defined  and  relatively  slow-moving  threats. 

The  necessary  entrainment  of  thought  that  characterises  experts  and  expertise  within 
say  a  military,  technical  or  scientific  community  is  ideal  for  exploitation  of  the 
knowledge  in  that  community.  But  this  characteristic  makes  it  unsuited  to  exploring 
outside  that  space  for  innovative  solutions  and  ideas.  The  empirically  known  and 
knowable  domains  are  of  crucial  importance  in  exploiting  the  knowledge  of  the 
intelligence  community,  but  they  need  to  be  supplemented  and  fed  by  new  exploratory 
ways  of  making  sense  of  the  world  that  are  not  tied  to  existing  mental  models  and 
paradigms  based  on  order,  process,  consensus  and  best  practice. 

Dealing  with  the  apparent  duality  of  the  problem  space  will  require  a  shift  in 
institutional  thinking  that  appears  problematic,  but  once  made  will  reveal  gains  and 
benefits.  In  this  sense,  we  contend  that  much  of  the  extant  intelligence  procedures, 
doctrine,  processes  and  products  are  a  part  of  ordered-domain  thinking,  and  as  such 
need  to  be  complemented  -  not  replaced  -  by  ideas,  mental-models,  techniques  and 
tools,  informed  by  the  un-ordered  domain  of  sense-making. 

Applying  ordered-space  thinking  to  inherently  un-ordered  domains  is  a  recipe  for 
failure.  Stewart  (2002)  describes  a  group  of  marines  taken  to  the  New  York  Mercantile 
Exchange  in  1995  to  be  taught  and  to  play  with  simulators  of  the  trading  environment. 
Naturally  the  traders  won  each  time.  But  when  the  traders  visited  the  Marine  Corp's 
base  in  Quantico  and  played  war  games  against  the  marines,  they  won  yet  again.  What 
they  realised  is  that  the  traders  were  skilled  at  comprehending  patterns  and 
intervening  to  favorably  exploit  those  patterns.  The  Marines,  on  the  other  hand,  like 
most  business  school  graduates,  had  been  trained  to  collect  and  analyse  data  and  then 
make  rational  decisions.  In  a  dynamic  and  constantly  changing  environment,  it  is 
possible  to  achieve  some  degree  of  comprehension  of  un-order  but  not  to  assume 
order6. 

2.6  Summary 

Table  1  characterises  the  space  of  sense  making  by  comparing  and  contrasting  two 
approaches  to  sense  making,  based  around  the  main  concepts  referenced  in  this  paper 
-  knowledge,  and  sense  making  in  complex  environments. 


6  In  another  experiment  a  group  of  West  Point  graduates  was  asked  to  manage  the  playtime  of  a 
kindergarten  as  a  final  year  assignment.  The  cruel  thing  is  that  they  were  given  time  to  prepare.  They 
planned;  they  rationally  identified  objectives;  they  determined  backup  and  response  plans.  They  then  tried 
to  “order”  children’s  play  based  on  rational  design  principles,  and,  in  consequence,  achieved  chaos.  They 
then  observed  what  teachers  do.  Experienced  teachers  allow  a  degree  of  freedom  at  the  start  of  the 
session,  then  intervene  to  stabilise  desirable  patterns  and  destabilise  undesirable  ones;  and,  when  they  are 
very  clever,  they  seed  the  space  so  that  the  patterns  they  want  are  more  likely  to  emerge. 
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Table  1:  Old  and  New  Sense  Making 


"Old"  Sense  Making 

"New"  Sense  Making 

Based  on  19th  century  physics  (equilibrium,  stability, 
deterministic  dynamics) 

Based  on  biological  metaphors  (structure,  patterns, 
self -organisation,  life  cycle) 

Sees  the  world  as  orderly,  predictable  and  well- 
understood 

Sees  the  world  as  complex,  unpredictable  and 
poorly  understood 

Knowledge  can  be  codified,  centralised  and 
managed 

Knowledge  resides  largely  in  the  minds  of  people 

Information  is  key  to  better  sense  making 

Knowledge  is  key  to  better  sense  making 

Sense  making  occurs  within  a  well-defined 
organisational  hierarchy 

Sense  making  occurs  across  and  beyond  defined 
organisational  boundaries 

Teams  actively  seek  confirmation  of  their  views 

Teams  actively  seek  refutation  of  their  views  and  a 
diversity  of  views 

Key  enabler  is  networking  of  IT  systems 

Key  enabler  is  networking  of  people  and  software 
agent  systems 
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3.  Implications 

A  key  implication  for  the  intelligence  community  is  that  any  view  of  reality  is  likely  to 
be  coloured  and  influenced,  to  a  greater  or  lesser  degree,  by  individual  or 
organisational  perceptions  -  their  existing  patterns  of  interpretation,  implicit 
assumptions,  and  beliefs.  As  a  result,  intelligence  products  that  attempt  to  describe  and 
represent  real  world  events  and  situations  are  inherently  (and  generally 
unintentionally)  coloured  by  the  individual  or  organisation  that  produces  them. 

For  intelligence  production,  this  means: 

•  All  intelligence  analysis  is  biased  by  the  individual  analyst's  or  organisation's 
perception  and  their  existing  patterns  of  interpretation,  implicit  assumptions, 
and  beliefs.  No  intelligence  analysis  of  a  complex  issue,  then,  can  ever  be  free  of 
this  bias; 

•  What  an  analyst  or  an  intelligence  organisation  considers  important,  relevant 
and  worthy  of  consideration  within  the  intelligence  domain  is  also  subject  to 
the  same  bias,  as  a  result  identifying  new  or  unexpected  (relative  to  the  analysis 
or  the  organisation's  past  experiences)  threats,  situations,  or  events  is  very 
difficult.  They  simply  may  not  be  seen; 

•  All  information  is  interpreted  by  what  is  already  known.  Analysts  and 
intelligence  organisations  don't  deliberately  look  for  information  to  refute  or 
contradict  what  is  already  known.  If  they  incidentally  discover  contradictory 
information,  it  may  simply  be  ignored  (Heuer  1999);  and 

•  Analysts  require  a  commitment  to  challenge,  refine  and  challenge  again  their 
working  hypotheses  and  assumptions. 

For  consumers  of  intelligence  products,  this  means: 

•  Intelligence  products  are  not  objective  facts,  but  high  quality  conjecture  and 
opinion  describing  events;  and 

•  Intelligence  products  are  ultimately  created  within  a  context.  This  context 
includes  the  analyst's  or  organisation's  intention  for  the  use  of  the  product,  and 
the  assumptions,  world  view,  patterns  of  thinking,  and  so  on,  that  all  underpin 
the  product.  As  a  result,  to  truly  understand  intelligence  the  context  in  which  it 
was  created  needs  to  be  understood. 

The  Flood  Report  (Flood  2004,  Chapter  6)  deals  in  detail  with  the  need  for 
contestability  of  assessments.  "For  Australia  to  have  the  highest  quality  assessment, 
analysts  need  to  be  challenged,  confronted  by  different  perspectives,  and  alerted  to 
flaws  in  their  arguments"  (Flood  p.  84).  The  means  for  achieving  contestability  are 
outlined  at  a  high  level  in  the  report,  including  the  need  for  contestability  between  DIO 
and  ONA.  However,  a  conceptual  basis  for  sense  making  is  also  needed  in  this  regard. 
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along  with  informed  advice  on  how  this  approach  impacts  intelligence  practice  and 
doctrine. 

The  next  two  sections  discuss  the  implications  of  these  ideas  in  terms  of  the  tools  and 
organisational  structures  employed  within  the  intelligence  community.  Together  they 
can  be  seen  as  part  of  an  ongoing  organisational  process  of  mindfulness  towards 
analytic  failure  (and  success)  and  as  a  hedge  against  institutionalised  ways  of  thinking 
and  modes  of  behaviour. 

3.1  Tools  for  Thinking 

Much  of  the  current  focus  in  the  intelligence  capability  development  domain  is  on 
improved  information  management;  this  has  produced  an  emphasis  on  data 
search/ retrieval,  pattern  analysis  and  visualisation  technologies.  However,  future 
development  of  cognitive  support  tools  to  add  knowledge  exploitation  and  sense  making 
within  the  intelligence  domain  must  go  further  and  will  need  to  encompass: 

•  Hypothesis  testing  and  "what-if”  scenario  testing  tools.  These  tools  allow  analysts  to 
explore  and  test  out  different  hypotheses  and  theories  about  the  world  to 
challenge  assumptions  and  explore  alternative  interpretations.  As  an  example 
Pope  and  Josang  (2005)  have  recently  developed  a  formal  approach  to  the 
evaluation  of  competing  hypotheses,  based  on  a  calculus  known  as  Subjective 
Logic,  that  allows  for  integration  of  empirical  and  statistical  data  as  well  as  for 
judgments  made  by  analysts. 

•  Sophisticated  applications  that  allow  analysts  to  fully  explore  the  data  used  in  sense 
making.  Although  pattern/ trend  analysis  has  been  the  traditional  realm  of 
intelligence  applications,  future  developments  need  to  migrate  from  merely 
identifying  patterns/ trends  to  enabling  analysts  to  better  comprehend  patterns  in 
their  data,  build  mental  models  of  their  data,  and  to  structure  and  re-structure 
their  data.  These  would  include  sophisticated  data/ information  query  tools  and 
data/ information  visualisation  tools  which  could  employ  expert  systems, 
artificial  neural  networks  and  machine  learning  technologies. 

As  well  as  tools  to  support  the  production  of  intelligence,  apparatus  to  capture  and 
embed  the  context  that  surrounds  the  intelligence  product  is  also  needed.  These  tools 
allow  the  creators  of  intelligence  products  to  include  the  processes,  assumptions,  and 
world  views  used  to  create  the  intelligence.  These  same  descriptions  can  be  used  by  the 
consumers  of  intelligence  products  to  build  a  richer  understanding  of  the  intelligence 
products  they  are  using. 

While  the  focus  of  this  paper  is  on  the  largely  human  capabilities  involved  in  sense 
making,  it  is  worth  pointing  out  in  this  section  the  need  for  information  systems  that 
underpin  the  types  of  cognitive  support  tools  under  discussion.  There  is  a  clear  need 
for  a  whole-of-enterprise  approach  to  managing  and  using  information  that  is  all  too 
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frequently  locked  away  in  stove-piped  applications  that  make  it  difficult  to  share  data 
with  other  systems.  In  addition  the  ownership  and  control  of  these  applications  is  often 
within  one  agency  making  sharing  and  access  across  organisational  boundaries 
problematic.  While  reliable  and  secure  access  to  data  is  improving,  it  is  in  the  logical 
description  and  transformation  of  information  -  not  just  its  transport  -  that  the  main 
barriers  to  enterprise-wide  use  of  information  exist.  As  stated  in  the  US  Government 
Semantic  Interoperability  Community  of  Practice  white  paper  (SICoP  2005): 

"The  goal  is  not  just  to  connect  systems,  but  also  to  make  the  data  and  information 
resident  within  these  systems  interoperable  and  accessible  for  both  machine 
processing  and  human  understanding.  " 

Computing  initiatives  such  as  the  Semantic  Web7  offer  a  way  ahead  in  this  regard  as 
do,  in  the  shorter  term,  the  Service  Orientated  Architecture  (SOA)  approach  to 
managing  and  using  complex  information  environments  (see  for  example 
http://www.service-architecture.com/).  Web  Services  are  an  implementation  of  the 
SOA  based  around  commonly  used  internet  protocols  such  as  HTTP  and  data 
definition  languages  such  as  XML.  They  provide  a  practical,  open,  standards-based 
approach  to  whole-of -enterprise  information  management. 

In  the  longer  term  one  goal  for  the  intelligence  community  is  to  spend  less  time  finding 
information  and  more  time  analysing  it.  More  advanced  software  including 
"intelligent"  agents  that  use  a  semantic  computing  infrastructure  will  help  to  automate 
a  lot  of  the  drudge  work  of  information  gathering,  and  will  play  a  part  in  this  vision. 

3.2  Implications  for  Community  and  Collaboration 

Underlying  patterns  of  interpretation,  implicit  assumptions,  and  beliefs  can  be  a  major 
problem  for  the  manufacture  of  intelligence  products  if  they  are  not  recognised  and 
managed.  However,  when  combined,  several  different  views  of  reality  are  likely  to  be  a 
closer  approximation  to  reality  then  a  single  view;  exploited  in  this  way,  such  a  range 
of  views  can  be  a  major  asset  for  the  manufacture  of  intelligence  products.  Providing 
organisational  structures  and  toolsets  that  assist  in  reconciling  and  generating  different 
views  may  be  seen  as  constituting  a  powerful  form  of  risk  management  for  the  ADO 
intelligence  community. 

Key  to  supporting  peer  review  is  providing  opportunities,  structures  and  technology 
that  allow  individual  analysts  to  interact  with  each  other,  sharing  their  underlying 
theories  and  ideas  and  examining  and  testing  out  each  other's  theories  and  ideas. 

To  do  this  the  intelligence  community  requires  an  environment  where  different  views 
are  encouraged,  and  exploited  to  produce  full,  multifaceted  analysis.  In  such  an 


7  See  The  World  Wide  Web  Consortium’s  Semantic  Web  Activity  page: 
<http://www.w3.  org/2001/sw>/ 
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environment  there  are  organisational  structures  that  support  collaboration  across 
group  and  organisational  boundaries8. 

Cynefin's  multi-ontology  sense  making  framework  (Snowden  2005)  discusses  a  means 
of  achieving  a  requisite  level  of  diversity  in  both  the  ways  we  interpret  the  world  and 
the  way  we  act  in  it.  Requisite  diversity  means  ensuring  the  acceptance  of  a  sufficient 
level  of  divergence  to  enable  the  sensing  of  weak  signals  (terrorist  threat  or  operational 
opportunity)  and  avoidance  of  pattern  entrainment  of  past  success,  while  maintaining 
a  sufficient  focus  to  enable  decisive  and  appropriate  action. 

In  this  respect  there  are  two  important  types  of  collaborative  networks: 

Community  of  Practice.  Having  experts  in  particular  areas  within  the  community, 
whether  that  area  is  a  country,  a  religion,  or  a  terrorist  organisation,  is  crucial  to  the 
quality  of  intelligence  produced  by  the  community.  The  Community  of  Practice 
(CoP)  is  centred  on  a  well-defined  domain  of  knowledge  and  expertise;  it  taps  into 
depth  of  specialist  knowledge  reflecting  an  environment  of  'conventional  wisdom'. 
The  members  of  the  community  share  a  common  set  of  patterns  of  interpretation, 
implicit  assumptions,  and  beliefs. 

The  goal  of  the  community  is  to  create,  maintain,  and  share  its  knowledge  within  a 
well-defined  domain.  These  communities  can  be  informal  or  formally  structured, 
and  are  typically  long-lived  in  comparison  to  task  or  team  oriented  groupings.  All 
members  can  equally  share  the  community's  knowledge,  and  equally  add  to  the 
community's  knowledge  through  their  work  and  experiences.  These  kinds  of 
communities  also  perform  the  role  of  enculturation  of  new  members  into  the 
knowledge  of  the  community  -  passing  on  the  facts,  methods,  information,  the  lore, 
the  language,  and  the  ways  of  thinking  that  are  a  part  of  the  community  (Wenger 
1998).  This  can  be  seen  as  an  ordered  domain  process  that  incrementally  grows  and 
maintains  knowledge.  It  roughly  maps  onto  the  top  right  of  the  sense  making 
domains  of  Table  2. 

Exploration  Network  (EN).  Increasingly  important,  this  relates  to  the  formation  of 
new  patterns  of  perceptions,  new  ways  of  understanding  the  world,  the  disruption 
of  existing  beliefs  and  ultimately  innovation  across  the  entire  enterprise.  In  effect  it 
seeks  to  tap  into  breadth  of  knowledge  to  create  an  environment  encouraging 
counter-intuitive  insight. 

An  example  of  an  exploration  network  is  a  (generally  informal)  assemblage  of 
friends,  associates  and  colleagues  drawn  from  inside  and  outside  the  functional 


s  These  communities  are  supported  by  technology  such  as  basic  tools  to  support  collaboration  across  time 
and  space.  These  include:  communications  tools  (e-mail,  instant  messaging  and  so  on);  tools  to  support 
sharing  of  information;  and  tools  and  processes  that  allow  individuals  to  discover  each  other,  and  to  build 
working  relationships  with  each  other. 
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groupings  of  task  and  organisation.  Membership  of  these  kinds  of  communities  is 
loosely  defined,  with  members  having  similar  or  very  different  patterns  of 
interpretation,  assumptions,  and  beliefs.  Potentially  these  networks  can  work  at  the 
edges  of  what  is  known  where  existing  patterns  of  interpretation,  implicit 
assumptions  and  beliefs  fail.  We  argue  that  these  networks  need  to  be  recognised, 
cultivated  and  exploited  by  the  intelligence  enterprise.  The  exploration  network 
grows  new  patterns  of  perception  that  may  be  exploited  within  the  rest  of  the 
intelligence  community,  and  roughly  maps  onto  the  top  left  of  Figure  1. 

Surowiecki  (2004)  argues  compellingly  that  in  many  situations  involving  cognition,  a 
group  of  people  is  smarter  than  the  smartest  people  in  that  group.  Examples  of 
collective  intelligence  include: 

•  The  group  average  estimate  of  the  number  of  jellybeans  in  a  jar  almost  always  is 
more  accurate  than  the  best  individual  guess. 

•  In  TV  game  shows  the  collective  audience  answers  often  do  better  than  those  of 
a  chosen  "expert". 

According  to  Surowiecki,  collective  intelligence  relies  on: 

•  Diversity  of  view  within  the  group 

•  Independence  of  agents  within  the  group 

•  De-centralisation  of  the  group 

•  A  means  for  aggregating  opinions 

The  similarity  between  this  type  of  group  and  the  exploration  network  described  above 
is  striking.  The  necessary  facets  of  collective  intelligence  identified  by  Surowiecki  allow 
the  intelligence  community  to  construct  these  networks  as  part  of  their  cognitive 
toolkit,  though  the  issue  of  security  needs  to  be  risk  managed. 

The  exploration  and  CoP  networks  are  compared  and  contrasted  in  Table  2.  These 
networks  serve  different  purposes  and  have  different  sense  making  methodologies  and 
approaches.  For  the  intelligence  community  the  CoP  helps  to  maintain  and  build  deep 
knowledge  in  a  particular  area  or  domain.  The  EN  is  designed  to  generate  new 
conjectures  and  confront  existing  ideas  and  conceptualisations. 

One  of  the  big  challenges  here  is  management  of  this  type  of  network9.  The  intelligence 
community  needs  to  be  able  to  tap  into  the  knowledge  of  its  various  agencies,  wider 
Governmental  departments  and  into  the  diverse  Australian  public  for  alternate 
viewpoints  and  theories.  Finding  people  in  the  wider  community  who  act  as  bridges 
between  sub-networks  or  localised  groupings  is  important,  as  is  finding  techniques 
that  facilitate  learning  and  innovation  around  these  networks. 

Critical  to  the  development  of  an  exploration  network  to  complement  existing  and 
well-established  CoPs  is  exploiting  the  growing  field  of  social  network  analysis  (SNA). 


9  In  contrast  to  a  CoP  where  membership  is  reasonably  well-defined  based  on  professional,  technical  or 
organisational  duties,  in  an  EN  participation  in  the  network  is  much  more  uncertain  and  diverse.  In  fact 
diversity  of  view  and  opinion,  serendipitous  encounters  and  a  bottom-up  organisational  structure  are  seen 
as  important  elements  of  an  EN. 


17 


DSTO-GD-0440 


This  discipline  provides  an  important  scientific  basis  for  comprehending  the  nature  of 
collaboration  within  the  multi-dimensional  network  that  is  the  intelligence  community 
as  well  as  the  multiple  interdependent  ways  of  being  connected.  In  this  respect, 
affiliation  networks  and  the  existence  of  the  small-world  phenomenon10  provides  a 
potentially  useful  mechanism  for  extending  the  connectedness  and  reach  of  the 
intelligence  enterprise.  This  aspect  of  collaboration  warrants  further  research, 
especially  given  the  migration  towards  a  network  centric  focus  for  the  Australian 
Defence  Organisation. 

Table  2:  Community  of  Practice  and  Exploration  Network  characteristics 


Community  of  Practice 

Exploration  Network 

Specialised  terminology 

Everyday  language 

High  levels  of  abstraction 

Low  levels  of  abstraction 

Shared  practice  and  domain  of  interest 

Shared  experiences,  values  and  beliefs. 

Well-defined  practice  within  the  domain  -  the  set  of 
frameworks,  tools,  information,  language  and 
documents  that  the  community  shares 

The  development  of  a  practice  is  a  possible,  long¬ 
term  outcome  of  exploration,  not  a  given 

Well-defined  areas  of  common  interest  (the  domain 
of  the  CoP) 

Often  poorly  defined  areas  of  common  interest 

Long-lived,  relatively  static  membership 

Short-lived,  dynamic  associations 

Community  members  defined  by  professional  or 
organisational  groupings 

Networks  form  and  re-form  depending  on  task  and 
need 

Goal  is  incremental  improvement  in  applying 
knowledge  in  a  well-defined  area 

Goal  is  to  develop  new  interpretations,  conjectures, 
ideas  and  ways  of  looking  at  the  world  that  may  be 
exploited  for  a  purpose 

Examples  include  guilds,  scientists  within  a  field, 
technical  repair  staff,  software  engineers,  and 
intelligence  experts  in  a  particular  field 

Examples  include  heterogeneous  work  units  such 
as  Tiger  Teams,  and  social  networks  such  as 
Community  Action  Groups 

10  This  is  the  widely  reported  finding  that  though  there  are  6  billion  people  on  Earth,  there  are  no  more 
than  6  degrees  of  separation  between  any  two  of  them. 
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4.  The  Military  Enterprise  Perspective 

This  section  places  the  ideas  of  the  foregoing  sections  into  a  wider  framework  and 
looks  at  how  sense  making  fits  into  the  future  military  enterprise  on  a  broader  scale,  by 
drawing  on  Haeckel's  Sense  and  Respond  strategy  and  Lambert  and  Scholz's  UC2 
framework  (Lambert  and  Scholz  2005). 

Haeckel  formulates  a  response  to  a  world  of  increasingly  rapid  and  unpredictable 
change  in  Adaptive  Enterprise  (1999).  He  writes  in  the  Introduction: 

"The  only  kind  of  strategy  that  makes  sense  in  the  face  of  un-predictable  change  is 
a  strategy  to  become  adaptive." 

Adaptive  Enterprise  outlines  the  systematic  means  by  which  large,  complex 
organisations  can  successfully  adapt  to  change.  Though  written  from  a  business 
perspective  the  ideas  have  relevance  to  the  ADO.  The  vision  of  moving  from  a  plan  to 
produce  specific  offerings  for  specific  markets  to  a  structure  for  sensing  and  responding 
to  change  faster  than  the  competition  has  resonance  for  military  operations  where  the 
customer  is  often  an  adversary,  and  where  a  plan  for  action  is  better  expressed, 
particularly  in  complex  environments,  as  an  adaptive  design  for  action. 

Renaming  Boyd's  OODA  loop  as  the  adaptive  SIDA  (Sense,  Interpret,  Decide,  Act) 
loop,  Haeckel  identifies  the  first  two  elements  collectively  with  Sensing,  and  the  last 
two  with  Responding.  Much  of  the  sense  making  discussion  in  the  previous  sections  fits 
into  to  the  Sensing  portion  of  the  decision  loop  and  raises  the  question  of  how  it  relates 
more  generally  to  the  adaptive  enterprise. 

In  the  systems  approach  to  adaptivity  outlined  by  Haeckel11  there  are  five  core 
enterprise  competencies: 

•  Knowing  earlier  (Sensing  and  Interpreting) 

•  Managing  by  Wire  (Augmented  Sensing  and  Responding) 

•  Designing  an  organisation  as  a  system  (Strategic  Context) 

•  Dispatching  capabilities  from  the  customer  request  back  (Tactical  Coordination) 

•  Leadership  by  Context  and  Coordination 

Many  of  the  enablers  of  better  sense  making  outlined  in  Section  3  can  be  viewed  as 
means  to  "know  earlier"  -  by  improving  the  peripheral  vision  of  the  enterprise, 
allowing  weak  signals  to  be  interpreted  and  used  as  a  basis  for  understanding  and 
action.  The  UC2  framework,  under  an  aegis  focussed  on  "achievable  intent  through 
unity  with  diversity",  provides  an  outline  of  many  of  the  other  core  competencies  for 
the  ADF  as  it  moves  towards  Network  Centric  Warfare.  In  particular  it  addresses  the 
design  requirements  of  an  adaptive  ADF  by  looking  at  capability  development  through 
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a  model  of  partial  design  contracting,  and  discusses  how  Command  and  Control  is 
better  characterised  by  coordination  of  multiple  agencies  (including  subordinate 
military  commanders  and  their  staff)  to  make  good  and  mutually  coherent  local 
decisions  within  the  context  of  an  overall  intent. 

Of  particular  interest  is  the  "agreement  network"  that  underpins  the  UC2  concepts.  If 
the  EN  and  CoP  networks  introduced  in  Section  3.2  are  part  of  the  "Sensing" 
apparatus,  then  this  network  can  be  viewed  as  part  of  the  "Respond".  For  an 
operational  mission,  the  agreement  network  is  a  dynamic  association  of  agent 
capabilities,  which  include  software  systems  as  well  as  people  and  equipment,  brought 
together  to  achieve  a  purpose  on  the  basis  of  the  competencies  and  roles  of  the  agents 
involved.  Mission  agreement  is  facilitated  with  social  coordination  protocols  that 
encapsulate  the  roles  and  accountabilities  of  these  dynamic  societies. 
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5.  Conclusions 


Since  the  end  of  the  Cold  War  estimative  intelligence  in  particular  has  shifted  from 
dealing  with  what  was  often  assumed  to  be  a  knowable  and  reasonably  well-defined 
and  well-understood  objective  reality  to  one  in  which  reality  is  fundamentally 
complex,  uncertain,  and  may  not  be  not  fully  understandable.  The  edge  over  an 
adversary  is  now  more  in  the  analytical,  predictive  and  cognitive  abilities  that  can  be 
brought  to  bear  on  the  mass  of  information  than  on  the  collection  of  the  information. 
This  has  important  capability  implications  for  developing  a  balance  between  sensing 
(capacity  to  observe)  and  sense  making  (capacity  to  orientate)  in  the  context  of  the 
ADO's  migration  towards  network-enabled  operations. 

This  paper  has  argued  that  intelligence  is  a  cognitive  activity,  and  that  sense  making  is 
a  powerful  way  of  conceptualising  the  cognitive  activities  performed  by  intelligence 
analysts. 

Within  this  view  of  the  intelligence  activity,  intelligence  generated  from  sense  making 
is  informed  conjecture.  This  raises  a  number  of  implications  for  the  crucial  human 
collaborative  networks  that  support  this  process  and  the  systems  that  supply 
information  to  it. 

Core  to  improving  the  cognitive  activity  of  intelligence  analysis  are  tools  for  conjecture 
testing  and  analysis,  and,  in  the  longer-term,  machines  that  are  capable  of 
understanding  the  meaning  of  information  they  have  access  to. 

Collaborative  structures  that  relate  to  the  formation  of  new  patterns  of  perceptions, 
new  ways  of  understanding  the  world,  and  the  disruption  of  existing  mind-sets  are 
needed.  Such  structures  seek  to  tap  into  breadth  of  knowledge  in  and  beyond  the  ADO 
intelligence  community  to  create  an  environment  encouraging  counter-intuitive 
insights. 

The  relation  of  an  augmented  capability  for  sensing  and  understanding  to  emerging 
whole-of-enterprise  concepts  such  as  Sense  and  Respond  and  UC2  is  worthy  of  more 
detailed  consideration. 


21 


DSTO-GD-0440 


References 

Breton,  R.  and  Rousseau,  R.  (2005),  "  The  C-OODA:  A  Cognitive  Version  Of  The  OODA 
Loop  To  Represent  C2 Activities" .  10th  CCRTS:  The  Future  of  Command  and  Control. 

Conklin,  J.,  (2004).  Dialog  Mapping:  Making  Sense  of  Project  Fragmentation,  Chapter  2. 
forthcoming.  See:  http:/ /www.cognexus.org 

Davenport,  T.  H.  and  L.  Prusak  (1997).  Working  Knowledge:  How  Organizations  Manage 
What  They  Know,  Harvard  Business  School  Press. 

Endsley,  M.  (1995).  Toward  a  theory  of  situation  awareness  in  dynamic  systems. 
Human  Factors,  37(1),  32-64 

Flood,  P  (2004).  Report  of  the  Inquiry  into  Australian  Intelligence  Agencies,  Australian 
Government  Publication,  Online  at 

http:/ / www.pmc.gov.au/ publications/intelligence_inquiry/ docs/intelligence_report. 
pdf. 

Haeckel,  S.  H.  (1999).  Adaptive  Enterprise:  Creating  and  Leading  Sense-and-Respond 
Organizations,  Harvard  Business  School  Press. 

Heuer,  R.  J.  (1999).  Psychology  of  Intelligence  Analysis.  Washington,  DC,  Center  for  the 
Study  of  Intelligence,  Central  Intelligence  Agency. 

Kauffman,  S  (1996).  At  Home  in  the  Universe:  The  Search  for  Laws  of  Self-Organization  and 
Complexity,  Oxford  University  Press. 

Klein,  G.  A.  (1993).  A  recognition-primed  Decision  (RPD)  model  of  rapid  decision 
making.  In  G.  Klein,  J.  Orasanu,  R.  Calderwood,  &  C.E.  Zsambok  (Eds),  Decision 
making  in  action:  Models,  and  methods  (pp.  138-147).  Norwood,  NJ:  Ablex. 

Kurtz,  C.  F.  and  D.  J.  Snowden  (2003).  "The  new  dynamics  of  strategy:  Sense-making  in  a 
complex  and  complicated  world."  IBM  Systems  Tournal  42(3). 

Lambert,  D.  and  Scholz,  J.  (2005).  "A  Dialectic  for  Network  Centric  Warfare,  "  10th  CCRTS: 
The  Future  of  Command  and  Control. 

Leedom,  D.  (2001).  Sensemaking  Symposium.  Command  and  Control  Research 
Program.  Online  at 

www.dodccrp.org/  events/ 2001/ sensemaking_symposium/ docs/ FinalReport/ Sense 
making_Final_Report.pdf 


22 


DSTO-GD-0440 


McDermott,  R.  (1999).  "Learning  Across  Teams:  How  to  build  communities  of  practice  in 
team  organisations."  Knowledge  Management  Review  May/June(8). 

Nitecki,  J.  (1985).  "The  concept  of  information-knowledge  continuum:  implications  for 
librarianship."  Journal  of  Library  History  20(4):  387-407. 

Nye,  J.  (1994).  "Peering  into  the  Future."  Foreign  Affairs  73(4). 

Polanyi,  M.  (1966).  The  Tacit  Dimension.  London,  Routledge  &  Kegan  Paul  Ltd. 

Pope,  S.  and  Josang,  A.  (2005),  Analysis  of  Competing  Hypotheses  using  Subjective  Logic, 
10th  CCRTS:  The  Future  of  Command  and  Control 


Snowden,  D.  (2002).  "Complex  acts  of  knowing:  paradox  and  descriptive  self-awareness." 
Journal  of  Knowledge  Management  6(2). 

Snowden,  D.  (2005). " Multi-ontology  sense  making:  a  new  simplicity  in  decision  making". 
Management  Today,  Yearbook  2005,  Vol  20 

SICoP,  (2005). "  Introducing  Semantic  Technologies  and  the  Vision  of  the  Semantic  Web " 
KM.GOV  White  Paper,  editor:  Fromm,  K. 

Stacey,  R.  D.  (1992).  Managing  the  Unknowable:  Strategic  Boundaries  Between  Order  and 
Chaos  in  Organizations,  Jossey-Bass. 

Stewart,  T.  A.  (2002).  How  To  Think  With  Your  Gut.  Business  2.0. 

Surowiecki,  J.  (2004).  The  Wisdom  of  Crowds:  Why  the  Many  Are  Smarter  Than  the  Few  and 
How  Collective  Wisdom  Shapes  Business,  Economies,  Societies  and  Nations,  Doubleday. 

Tovstiga,  G.,  Odenthal,  S.  and  Goerner  S.  (2004),  Sense-Making  And  Learning  In  Complex 
Organizations:  The  String  Quartet  Revisited,  The  Fifth  European  Conference  on 
Organizational  Knowledge,  Learning  and  Capabilities,  Innsbruck,  Austria,  April  2-3, 
2004 

Weick,  K.  E.  (1995).  Sensemaking  in  Organizations,  Sage  Publications. 

Wenger,  E.  (1998).  " Communities  of  Practice:  Learning  as  A  Social  System."  Systems 
Thinker. 


23 


DISTRIBUTION  LIST 


Sense  Making  in  the  ADO  Intelligence  Community 
Mark  Burnett,  Pete  Wooding  and  Paul  Prekop 

AUSTRALIA 


DEFENCE  ORGANISATION 
Task  Sponsor 

Deputy  Chief  Defence  Scientist  Policy 

S&T  Program 

Chief  Defence  Scientist 

Deputy  Chief  Defence  Scientist  Policy 

AS  Science  Corporate  Management 

Director  General  Science  Policy  Development 

Counsellor  Defence  Science,  London 

Counsellor  Defence  Science,  Washington 

Scientific  Adviser  to  MRDC,  Thailand 

Scientific  Adviser  Joint 

Navy  Scientific  Adviser 

Scientific  Adviser  -  Army 

Air  Force  Scientific  Adviser 

Scientific  Adviser  to  the  DMO 

Information  Sciences  Laboratory 

Chief  Command  &  Control  Division 
Research  Leader  Command  Decision 
Environments  Branch 

Research  Leader  Information  Enterprises  Branch 

Research  Leader  Joint  Command  Analysis  Branch 

Research  Leader  Intelligence  Information  Branch 

Head  Human  Systems  Integration 

Head  Information  Exploitation 

Head  Effects-Based  Modelling  and  Analysis 

Head  Distributed  Enterprises 

Head  Joint  Operations  Analysis  and  Support 

Head  Command  Concepts  and  Architectures 

Head  Command  Process  Integration  and  Analysis 

Head  Intelligence  Analysis 

Head  Information  Systems 

Mark  Burnett  (Author) 

Pete  Wooding  (Author) 

Paul  Prekop  (Author) 

Publications  and  Publicity  Officer,  C2D/EOC2D 

DSTO  Library  and  Archives 

Library  Edinburgh 
Library  Canberra 
Defence  Archives 


No.  of  copies 
1  printed 


1 

1 

1 

1 

Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
1 

Doc  Data  Sheet  &  Dist  List 
Doc  Data  Sheet  &  Dist  List 
Doc  Data  Sheet  &  Dist  List 
Doc  Data  Sheet  &  Dist  List 

Doc  Data  Sheet 

1 

1 

Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
1 

1  printed 
1  printed 
1  printed 

1  shared  copy 


1  copy  &  Doc  Data  Sheet 
Doc  Data  Sheet 
1 


Capability  Development  Group 

Director  General  Maritime  Development 

Director  General  Land  Development 

Director  General  Capability  and  Plans 

Assistant  Secretary  Investment  Analysis 

Director  Capability  Plans  and  Programming 

Director  General  Australian  Defence  Simulation  Office 


Doc  Data  Sheet 
1 

Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 


Chief  Information  Officer  Group 

Director  General  Australian  Defence  Simulation  Office 
AS  Information  Strategy  and  Futures 
Director  General  Information  Services 


Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 


Strategy  Group 

Director  General  Military  Strategy 
Assistant  Secretary  Governance  and 
Counter-Proliferation 


Doc  Data  Sheet 
Doc  Data  Sheet 


Navy 

Maritime  Operational  Analysis  Centre, 

Building  89/90  Garden  Island  Sydney  NSW 

Deputy  Director  (Operations) 

Deputy  Director  (Analysis) 

Director  General  Navy  Capability,  Performance  and  Plans, 
Navy  Headquarters 

Director  General  Navy  Strategic  Policy  and  Futures, 

Navy  Headquarters 


Doc  Data  Sht  &  Dist  List 
(shared) 


Doc  Data  Sheet 
Doc  Data  Sheet 


Air  Force 

SO  (Science)  -  Headquarters  Air  Combat  Group,  RAAF  Base,  Williamtown 

NSW  2314  Doc  Data  Sht  &  Exec  Summ 


Army 

ABCA  National  Standardisation  Officer  e-mailed  Doc  Data 

Land  Warfare  Development  Sector,  Puckapunyal  Sheet 

SO  (Science)  -  Land  Headquarters  (LHQ), 

Victoria  Barracks  NSW  Doc  Data  &  Exec  Summ 

SO  (Science),  Deployable  Joint  Force  Headquarters  (DJFHQ)  (L), 

Enoggera  QLD  Doc  Data  Sheet 


Joint  Operations  Command 

Director  General  Joint  Operations 

Chief  of  Staff  Headquarters  Joint  Operations  Command 
SO  Development  ADF  Warfare  Centre 
Director  General  Strategic  Logistics 
COS  Australian  Defence  College 


Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 


Intelligence  and  Security  Group 

AS  Concepts,  Capability  and  Resources  1 

DGSTA  Defence  Intelligence  Organisation  1  printed 

Manager,  Information  Centre,  Defence  Intelligence  Organisation  1 

Defence  Materiel  Organisation 

Deputy  CEO 

Head  Aerospace  Systems  Division 
Head  Maritime  Systems  Division 
Program  Manager  Air  Warfare  Destroyer 

OTHER  ORGANISATIONS 

National  Library  of  Australia 
NASA  (Canberra) 

UNIVERSITIES  AND  COLLEGES 
Australian  Defence  Force  Academy 

Library 

Head  of  Aerospace  and  Mechanical  Engineering 
Hargrave  Library,  Monash  University 


OUTSIDE  AUSTRALIA 

INTERNATIONAL  DEFENCE  INFORMATION  CENTRES 

US  Defense  Technical  Information  Center  1 

UK  Dstl  Knowledge  Services  1 

Canada  Defence  Research  Directorate  R&D  Knowledge 

&  Information  Management  (DRDKIM)  1 

NZ  Defence  Information  Centre  1 

ABSTRACTING  AND  INFORMATION  ORGANISATIONS 

Library,  Chemical  Abstracts  Reference  Service  1 

Engineering  Societies  Library,  US  1 

Materials  Information,  Cambridge  Scientific  Abstracts,  US  1 

Documents  Librarian,  The  Center  for  Research  Libraries,  US  1 

SPARES  5  printed 


1 

1 

Doc  Data  Sheet 


1 

1 


Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 
Doc  Data  Sheet 


Total  number  of  copies: 


Printed:  10  PDF:  25 


35 


Page  classification:  UNCLASSIFIED 

DEFENCE  SCIENCE  AND  TECHNOLOGY  ORGANISATION 

DOCUMENT  CONTROL  DATA  i-  privacy  marking/ caveat  (of  document) 

2.  TITLE  3.  SECURITY  CLASSIFICATION  (FOR  UNCLASSIFIED  REPORTS 

THAT  ARE  LIMITED  RELEASE  USE  (L)  NEXT  TO  DOCUMENT 
Sense  Making  in  the  Australian  Defence  Organisation  (ADO)  CLASSIFICATION) 

Intelligence  Community  ^  ^ 

Title  (U) 

Abstract  (U) 

4.  AUTHOR(S)  5.  CORPORATE  AUTHOR 

Mark  Burnett,  Pete  Wooding  and  Paul  Prekop  DSTO  Defence  Science  and  Technology  Organisation 

PO  Box  1500 

Edinburgh  South  Australia  5111  Australia 


6a.  DSTO  NUMBER  6b.  AR  NUMBER  6c.  TYPE  OF  REPORT  7.  DOCUMENT  DATE 

DSTO-GD-0440  AR-013-446  General  Document  July  2005 


8.  FILE  NUMBER 

9.  TASK  NUMBER 

10.  TASK  SPONSOR 

11.  NO.  OF  PAGES 

12.  NO.  OF  REFERENCES 

2005/1036540/1 

STR  02/124 

FASSP 

30 

26 

13.  URL  on  the  World  Wide  Web  14.  RELEASE  AUTHORITY 


http://www.dsto.defence.gov.au/corporate/reports/DSTO-GD-0440.pdf  Chief,  Command  and  Control  Division 

15.  SECONDARY  RELEASE  STATEMENT  OF  THIS  DOCUMENT 

Approved  for  public  release 

OVERSEAS  ENQUIRIES  OUTSIDE  STATED  LIMITATIONS  SHOULD  BE  REFERRED  THROUGH  DOCUMENT  EXCHANGE,  PO  BOX  1500,  EDINBURGH,  SA  5111 

16.  DELIBERATE  ANNOUNCEMENT 

No  Limitations 


17.  CITATION  IN  OTHER  DOCUMENTS  Yes 

18.  DEFTEST  DESCRIPTORS 

Intelligence  community 
Intelligence  analysis 
Cognitive  abilities 
Cognitive  processes 

19.  ABSTRACT 

This  paper  discusses  sense  making,  and  its  role  in  how  intelligence  analysts  understand  and  interpret  events. 
The  inherent  limitations  in  the  way  an  analyst,  based  on  organisational  and  personal  perspectives,  understands 
the  world  is  described  and  points  toward  the  need  for  meta-sense  making  techniques.  Insights  from  complex 
systems  theory  and  knowledge  management  can  be  used  to  understand  how  techniques  from  these  areas  can 
be  applied  in  practice,  and  assist  in  mitigating  some  of  the  risks  due  to  the  cognitive  limitations  inherent  in 
intelligence  processes.  The  relation  of  an  augmented  capability  for  sensing  and  understanding  to  emerging 
whole-of-enterprise  concepts  such  as  Sense  and  Respond  and  UC2  is  also  sketched. 


Page  classification:  UNCLASSIFIED 


